Web Survey Bibliography
Title Using eye-tracking to understand how fourth grade students answer matrix items
Author Maitland, A.; Sun, H.; Caporaso, A.; Tourangeau, R.; Bertling, J.; Almonte, D.
Year 2015
Access date 23.08.2016
Full text PDF (522 KB)
Abstract
Eye-tracking has been used to better understand the survey response process. For instance, eye-tracking has been used to identify questions that are difficult to comprehend, how to present long lists of response options, and to measure the length of fixation on definitions in Web surveys. Matrix questions have been commonly used in Web surveys as well as in other types of surveys.The literature shows respondents took less time to answer questions when they were presented in a matrix than when they were presented individually across separate pages or screens. The use of matrix questions, however, may also be associated with several undesirable outcomes, including higher breakoff rates, higher missing data rates, and straightlining. Relatively little is known about how children answer matrix questions. This paper demonstrates how eye-tracking was used to determine the feasibility of using matrix questions to measure the background characteristics of students in the National Assessment of Educational Progress (NAEP) questionnaire. Fourth grade students were answered both matrix and discrete versions of questions on tablet computers while wearing real-world eye tracking glasses. This study addresses four research questions related to the use of eye-tracking to test survey questions. First, we examine whether matrix items require more effort to answer than discrete items for fourth grade students. Second, we investigate how the processing of sub items change within a matrix. Third, we examine how the processing of questions change over time. In order to address these research questions, we examine difference in the mean number of fixations per word and the mean duration per word for matrix and discrete questions. Overall, the study finds support for the use of matrix questions with fourth grade students in the NAEP. Implications for the use of eye-tracking equipment to evaluate survey questions are also discussed.
Access/Direct link FCSM Research Conference Homepage (Abstract) / (Full text)
Year of publication2015
Bibliographic typeConferences, workshops, tutorials, presentations
Web survey bibliography - 2015 (291)
- Taking MARS Digital; 2015; Melton, E.; Krahn, J.
- A Comparison of the Effects of Face-to-Face and Online Deliberation on Young Students’ Attitudes...; 2015; Triantafillidou, A.; Yannas, P.; Lappas, G.; Kleftodimos, A.
- A Privacy-Friendly Method to Reward Participants of Online-Surveys; 2015; Herfert, M.; Lange, B.; Selzer, A.; Waldmann, U.
- Doing Online Surveys: Zum Einsatz in der sozialwissenschaftlichen Raumforschung; 2015; Nadler, R.; Petzold, K.; Schoenduwe, R.
- Are Fast Responses More Random? Testing the Effect of Response Time on Scale in an Online Choice Experiment...; 2015; Boerger, T.
- The impact of frequency rating scale formats on the measurement of latent variables in web surveys -...; 2015; Menold, N.; Kemper, C. J.
- Investigating response order effects in web surveys using eye tracking; 2015; Karem Hoehne, J.; Lenzner, T.
- Implementation of the forced answering option within online surveys: Do higher item response rates come...; 2015; Decieux, J. P.; Mergener, A.; Neufang, K.; Sischka, P.
- Internet Panels, Professional Respondents, and Data Quality; 2015; Matthijsse, S.; De Leeuw, E. D.; Hox, J.
- Self-administered Questions and Interviewer–Respondent Familiarity; 2015; Rodriguez, L. A., Sana, M., Sisk, B.
- Comparing Food Label Experiments Using Samples from Web Panels versus Mall Intercepts; 2015; Chang, L. C., Lin, C. T. J.
- Translating Answers to Open-ended Survey Questions in Cross-cultural Research: A Case Study on the Interplay...; 2015; Behr, D.
- The impact of gamifying to increase spontaneous awareness; 2015; Cape, P.
- Using eye-tracking to understand how fourth grade students answer matrix items; 2015; Maitland, A.; Sun, H.; Caporaso, A.; Tourangeau, R.; Bertling, J.; Almonte, D.
- Incentive Types and Amounts in a Web-based Survey of College Students; 2015; Krebs, C.; Planty, M.; Stroop, J.; Berzofsky, M.; Lindquist, C.
- Response Rates and Response Bias in Web Panel Surveys; 2015; Boyle, J.; Berman, L.; Dayton, Ja.; Fakhouri, T.; Iachan, R.; Courtright, M.; Pashupati, K.
- Characteristics of the Population of Internet Panel Members; 2015; Boyle, J; Freedner, N.; Fakhouri, T.
- Internet and Smartphone Coverage in a National Health Survey: Implications for Alternative Modes; 2015; Couper, M. P.; Kelley, J.; Axinn, W.; Guyer, H.; Wagner, J.; West, B. T.
- An Overview of Mobile CATI Issues in Europe; 2015; Slavec, A.; Toninelli, D.
- Using Mobile Phones for High-Frequency Data Collection; 2015; Azevedo, J. P.; Ballivian, A.; Durbin, W.
- Willingness of Online Access Panel Members to Participate in Smartphone Application-Based Research; 2015; Pinter, R.
- Who Has Access to Mobile Devices in an Online Opt-in Panel? An Analysis of Potential Respondents for...; 2015; Revilla, M.; Toninelli, D.; Ochoa, C.; Loewe, G.
- Who Are the Internet Users, Mobile Internet Users, and Mobile-Mostly Internet Users?: Demographic Differences...; 2015; Antoun, C.
- A Meta-Analysis of Breakoff Rates in Mobile Web Surveys; 2015; Mavletova, A. M.; Couper, M. P.
- The Best of Both Worlds? Combining Passive Data with Survey Data, its Opportunities, Challenges and...; 2015; Duivenvoorde, S.; Dillon, A.
- Optimizing the Decennial Census for Mobile – A Case Study; 2015; Nichols, E. M.; Hawala, E. O.; Horwitz, R.; Bentley, M.
- App vs. Web for Surveys of Smartphone Users: Experimenting with mobile apps for signal-contingent experience...; 2015; McGeeney, K.; Keeter, S.; Igielnik, R.; Smith, A.; Rainie, L.
- Using Video to Reinvigorate the Open Question; 2015; Cape, P.
- On the Go: How Mobile Participants Affect Survey Results; 2015; Barlas, F. M.; Thomas, R. K.
- The Matrix Lives On: Improving Grids for Online Surveys; 2015; Thomas, R. K.; Barlas, F. M.; Graham, P.; Subias, T.
- Variance Estimation for Surveys from Internet Panels ; 2015; Rivers, D.
- Sensitivity Analysis of Bias of Estimates from Web Surveys with Nonrandomized Panel Selection; 2015; Beresovsky, V.
- Detecting Fraud in a Survey Sample Recruited Online; 2015; Brown, D.; Dever, J. A.; Augustson, E.; Squiers, L.
- Survey Treatments and Response Modes: Bayesian Survival Analysis with Competing Risks; 2015; Minato, H.
- Purposefully Mobile: Experimentally Assessing Device Effects in an Online Survey ; 2015; Barlas, F. M.; Thomas, R. K.; Graham, P.
- Using equivalence testing to disentangle selection and measurement in mixed modes surveys ; 2015; Cernat, A.
- What do web survey panel respondents answer when asked “Do you have any other comment?”; 2015; Schonlau, M.
- On Climbing Stairs Many Steps at a Time: The New Normal in Survey Methodology; 2015; Dillman, D. A.
- Mobile Research Methods: Opportunities and challenges of mobile research methodologies. ; 2015; Toninelli, D. (Ed.); Pinter, R.; de Pedraza, P.
- Effect of Web-Based Versus Paper-Based Questionnaires and Follow-Up Strategies on Participation Rates...; 2015; Kilsdonk, E.; van den Heuvel-Eibrink, M. M.; van Dulmen-den Broeder, E.; van der Pal, H. J. H.; van...
- Polling Error in the 2015 UK General Election: An Analysis of YouGov’s Pre and Post-Election Polls...; 2015; Wells, A.; Rivers, D.
- Cell Phone and Face-to-face Interview Responses in Population-based Sur- veys - How Do They Compare?; 2015; Ghandour, L.; Ghandour, B.; Mahfoud, Z.; Mokdad, A.; Sibai, A. M.
- Collecting Health Research Data - Comparing Mobile Phone-assisted Personal Interviewing to Paper-and...; 2015; van Heerden, A. C.; Norris, S. A.; Tollman, S. M.; Richter, L. M.
- The Effects of Questionnaire Completion Using Mobile Devices on Data Quality. Evidence from a Probability...; 2015; Bosnjak, M.; Struminskaya, B.; Weyandt, K.
- Are Sliders Too Slick for Surveys? An Experiment Comparing Slider and Radio Button Scales for Smartphone...; 2015; Aadland, D.; Aalberg, T.
- Evaluation of an Adapted Design in a Multi-device Online Panel: A DemoSCOPE Case Study; 2015; Arn, B.; Klug, S.; Kolodziejski, J.
- Maximizing Data Quality using Mode Switching in Mixed-Device Survey Design: Nonresponse Bias and Models...; 2015; Axinn, W.; Gatny, H. H.; Wagner, J.
- Web Surveys Optimized for Smartphones: Are there Differences Between Computer and Smartphone Users?; 2015; Andreadis, I.
- Measuring Political Knowledge in Web-Based Surveys: An Experimental Validation of Visual Versus Verbal...; 2015; Munzert, S.; Selb, P.
- Validation of the new scale for measuring behaviors of Facebook users: Psycho-Social Aspects of Facebook...; 2015; Bodroza, B.; Jovanovic, T.